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Abstract 


During  the  past  decade  the  doctrine  of  Net  Centric  Warfare  has  emerged  and  grown. 
NCW  has  been  defined  as  an  infonnation  superiority-enabled  concept  of  operations  that 
generates  increased  combat  power  by  networking  sensors,  decision  makers,  and  shooters 
to  achieve  shared  awareness,  increased  speed  of  command,  higher  tempo  of  operations, 
greater  lethality,  increased  survivability,  and  degree  of  self  -synchronization.  (Alberts, 
Garstka,  and  Stein  (2000)).  Situational  Awareness  and  its  sharing  by  linked  warfighters  is 
thus  deemed  to  be  a  major  causative  factor  in  increasing  combat  power.  How  do  we 
create  and  measure  situational  awareness  and  relate  it  to  combat?  What  are  some  of  its 
determinants?  We  shall  see  in  the  results  of  the  controlled  warfighting  experiment 
examined  below  that  there  are  at  least  two  major  determinants  of  increased  Situational 
Awareness  for  a  warfighting  team,  viz.  use  of  a  relatively  complete  Common  Operational 
Picture  of  the  battlespace  and  time  spent  collaborating  with  this  COP  as  a  team. 

Approach 

In  the  approach  adopted  here,  variable  Situational  Awareness  was  experimentally 
created  in  a  team  context  in  a  series  of  Human-In-The-Loop  (HITL)  experiments  and,  via 
the  Common  Operational  Picture  (COP)  prototype,  related  to  combat  outcome.  In  the  first 
experiment  three  warfighting  teams  each  composed  of  a  retired  flag  officer  and  a  pair  of 
ship  captains  played  out  four  battles,  composed  of  two  versions  of  a  Persian  Gulf  air/sea 
combat  scenario  crossed  with  two  versions  of  infonnation  technology,  the  COP  prototype 
fed  by  national  and  organic  sensors  and  the  control  ensemble  with  the  ship  captains 
having  only  local  pictures  fed  by  organic  sensors  and  the  flag  officer  having  only  the 
national  sensor  fed  big  picture.  The  mission  of  the  teams,  including  the  two  ship  captains 
protecting  oil  platforms  in  the  Gulf,  was  to  identify  and  to  prosecute  advancing  enemy 
fast  attack  craft.  For  all  three  newly  created  teams  and  for  all  four  trials,  Situational 
Awareness  (SA)  was  defined  as  the  proportion  of  the  crisis  relevant  (or  mission  critical) 
set  of  warfighting  platforms  (red,  blue  or  neutral)  correctly  identified  as  important  by  the 
commander.  During  the  simulated  combat  operation,  using  the  RESA  wargame 
simulator,  the  commander’s  realization  of  the  situation,  his  Situational  Awareness,  was 
obtained  by  his  indication  on  a  map  of  the  platforms  he  deemed  important  at  that  time, 
i.e.  by  his  personal  Cognitive  Operational  Graphic  (COG).  The  commander’s  proportion 
correct  was  then  obtained  by  comparison  with  the  platfonns  on  the  simulator’s  Ground 
Truth  map  at  the  same  time.  Such  COG  measures  were  obtained  from  all  commanders  at 
the  end  of  each  of  the  three  phases  of  each  of  the  combat  scenarios.  Thus  greater  overlap 
between  the  commander’s  COG  and  Ground  Truth  was  indicative  of  greater  Situational 
Awareness  by  the  commander  at  the  time. 

We  hypothesized  (1)  that  use  of  the  COP  by  a  warfighting  team  causes  significant 
improvement  in  their  Situational  Awareness.  (Hiniker  &  Entin,  1990,  pg.  220)  Indeed, 
teams  using  the  COP  prototype  achieved  on  average  nearly  10  percent  better  Situational 
Awareness  than  control  teams,  i.e.  COP  teams  correctly  identified,  on  average  58  percent 
of  the  crisis  relevant  combat  platforms  throughout  the  combat  scenarios  compared  to  only 
53  percent  for  control  teams.  This  is  a  significant  difference  favoring  use  of  the  COP  (t  = 


3.06  for  5  degrees  of  freedom  with  paired  comparisons,  p  <  .05).  Based  upon  the  sample 
size  and  variability  in  these  results,  we  can  be  95  percent  confident  that  the  true 
difference  in  Situational  Awareness  between  the  two  technologies  is  positive  for  the  COP 
and  lies  between  1  percent  and  9  percent. 

More  recently,  there  has  been  expanded  work  on  the  concept  of  Situational  Awareness 
and  Shared  Awareness  (Perry,  Signori  &  Boon,  2004)  which  has  alerted  us  to  an 
additional  factor  among  the  causes  of  improved  Situational  Awareness,  and  for  which  we 
find  compelling  evidence  in  secondary  analysis  of  our  experimental  data.  This  variable, 
(T),  is  time  spent  by  a  warfighting  team  in  collaboration  with  a  COP,  complete  or 
incomplete.  Perry  et  al  define  Situational  Awareness  as  “  (a  warfighter’s)  realization  of 
the  current  situation  based  on  the  observed  CROP  (Common  Relevant  Operational 
Picture)”.  Similarly,  they  define  completeness  of  the  CROP  as  “the  degree  to  which  the 
information  is  free  of  gaps”;  and  correctness  as  “the  degree  to  which  information  agrees 
with  ground  truth.”  Our  original  approach  is  in  full  accord  with  this  terminology.  What  is 
added  here  is  their  plausible  hypothesis  on  “team  hardness”,  that  the  completeness  of  the 
(team)  system  for  recording  and  retrieving  information  (thus  improving  the  team’s 
Transactive  Memory  System  (TM(T))  depends  upon  how  frequently  the  team  has 
recently  collaborated  (T).  They  define  T  =  t  +  t,  where  t  represents  the  time  elapsed 
since  the  start  of  the  operation  and  x  represents  the  length  of  time  the  team  has  been 
training  or  operating  together.  (Perry,  Signore  &  Boon,  2004, pg.  79).  Their  model  treats 
the  strengthened  Transactive  Memory  System  as  a  major  contributor  to  Situational 
Awareness.  Both  t  and  t,  hence  T,  are  present  and  measurable  in  our  data  for  the  teams 
we  created  for  the  first  COP  HITL  experiment.  Thus,  we  hypothesize  (2)  that  increased 
time  spent  by  a  team  collaborating  with  the  COP  (T)  causes  increased  Situational 
Awareness  (SA)  in  team  members. 


Results 

Examination  of  Table  1,  below,  will  help  define  T  in  the  context  of  our  experiment. 

TABLE  1.  Degree  of  Situational  Awareness  for  Members  of  Warfighting  Teams 
under  COP  and  no-COP  Conditions  by  Time  Order  of  Observation. 


Sequence  of 

Observations  COP  no-COP  Delta 


1 1 

.56 

.48 

.08 

12 

.53 

.51 

.02 

13 

.58 

.51 

.07 

II  1 

.54 

.55 

-.01 

II  2 

.61 

.56 

.05 

II  3 

.65 

.56 

.09 

Average 

.58 

.53 

.05 

Each  of  the  12  data  points  in  the  table  is  an  average  of  the  18  Situational  Awareness 
scores  obtained  from  all  three  officers  from  all  three  teams  for  both  warfighting 
scenarios.  Since  each  team  played  all  four  conditions  (COP/not-COP,  Scenario 
A/Scenario  B),  each  team  used  the  COP  twice,  once  with  each  of  the  two  scenarios.  A 
team’s  first  use  was  termed  Replication  I  and  their  next  use,  Replication  II.  In  addition, 
within  each  Replication,  during  the  trial  scenario  run,  there  were  three  SA  data  takes  at 
roughly  30  minute  intervals,  yielding  the  measure  of  t.  Since  we  created  these  teams  from 
scratch  in  the  lab,  T  is  precisely  measurable  in  the  context  of  the  experiment.  Thus  in 
Replication  I,  T  =  E  f;  in  Replication  II,  T  =  t  +Z  f,  where  t  =  2  hr.,  from  experience  in 
the  previous  trial.  This  combination  of  two  Replications  and  three  scenario  time  phases 
results  in  the  ordering  of  data  presented  in  the  successive  rows  of  Table  One.  Thus  the 
Sequence  of  Observations  column  in  Table  One  corresponds  to  T  as  measured  by  time 
spent  working  with  teammates  using  the  COP  or  COP  surrogate  (not-COP).  The  values 
for  T  (in  minutes)  beginning  with  I  1  through  the  six  observation  points  to  II  3  are:  30, 

60,  105,  150,  180,  225  minutes  of  team  learning.  Figure  1  below  depicts  a  recasting  of 
the  results  for  Situational  Awareness  in  Table  1  while  incorporating  the  variable  T. 

Figure  1.  Degree  of  Situational  Awareness  by  Time  Using  COP  for  COP  and  Partial 

COP  Conditions 


The  results  depicted  in  Figure  1  provide  strong  support  for  Hypothesis  (2)  that  increased 
time  spent  by  a  team  collaborating  with  the  COP  (T)  causes  increased  Situational 
Awareness  (SA)  in  team  members.  For  the  two  curves  combined  (n  =  6),  the  overall 
correlation  between  T  and  SA  yields  r  =  .95,  p  =  .003.  Examining  the  correlation  within 
the  two  particular  technologies,  the  top  curve  smoothly  linking  the  data  points  for  SA  in 
teams  using  the  COP  for  increasing  T  affords  a  modestly  strong  correlation  (r  =  .76,  p  = 
.07)  between  SA  and  Time  collaborating  with  the  COP.  The  linear  equation  for  the  best- 
fit  line  is  SA  =  .52  +  .0004752  T,  with  a  Standard  Error  of  the  Estimate  of  .03.  The 
bottom  curve  smoothly  linking  data  points  for  SA  for  teams  in  the  not-COP  condition  for 
increasing  T  provides  an  even  stronger  correlation  (r  =  .95,  p  =  .004)  between  SA  and 


Time  using  the  (partial)  COP.  In  the  not-COP  or  Partial  COP  condition,  each  of  the  two 
ship  captains  had  only  his  local  tactical  picture  fed  only  by  his  ship’s  sensors  and  the 
admiral  had  only  his  big  picture  fed  by  national  satellites.  This  distributed  team  relayed 
information  to  each  other  via  voice  communications.  The  linear  equation  for  the  nearly 
parallel  best-fit  line  for  this  not-COP  condition  is  SA  =  .475  +  .000425  T,  with  a  Standard 
Error  of  the  Estimate  of  .0 1 . 

In  sum,  combining  the  findings  for  this  particular  experiment  we  find  a  simple  linear 
model  that  accounts  well  for  the  results.  We  might  term  this  the  Tech  Team  Model  of 
Situational  Awareness,  emphasizing  the  fact  that  we  are  examining  the  behavior  of  a 
socio-technical  system  where  the  COP  serves  as  a  kind  of  partner  in  the  team.  In  other 
words,  for  a  team  using  a  partial  assemblage  of  local  tactical  pictures,  begin  with 
Situational  Awareness  of  47.5  percent  of  relevant  battlefield  platforms;  for  a  team  using  a 
more  complete  COP,  add  4.5  per  cent,  yielding  an  initial  Situational  Awareness  of  52  per 
cent.  Thus  the  effect  of  the  relatively  complete  COP  is  to  lift  the  initial  level  of 
Situational  Awareness  of  team  members.  Then  for  a  team  equipped  with  either 
technology,  augment  their  Situational  Awareness  by  2.5  per  cent  for  every  hour  they 
spend  collaborating  as  a  team  with  the  COP.  Bear  in  mind  when  extrapolating  to  other 
situations  that  these  data  were  collected  over  a  short,  roughly  four  hour,  time  period.  For 
longer  time  periods,  the  damping  effect  of  the  standard  learning  curve  will  undoubtedly 
come  into  play. 

We  can  get  a  glimpse  of  some  more  general  considerations,  including  likely  learning 
curve  effects,  by  examining  the  Perry  model  in  more  detail.  Here  the  first  consideration 
is  to  relate  the  strength  of  the  transactive  memory  system,  TM  (T),  of  the  team  to  time 
spent  collaborating  with  the  COP,  T.  They  propose  the  logistic  curve  function,  otherwise 
known  as  the  learning  curve  or  growth  curve:  TM  (T)  =  1  -  e'aT,  where  a  is  a  steepness 
of  slope  parameter.  Figure  2  below  depicts  this  curve  for  our  time  frame  with  a  =  .01. 

We  selected  this  low  value  because  higher  values  provide  a  steeper  learning  curve  than 
seems  warranted  by  our  data. 


Figure  2.  Model  Relationship  between  Strength  of  Transactive  Memory  System 
(TM  (T))  and  Time  Spent  Collaborating  with  the  COP  (T) 


The  next  consideration  is  to  relate  Situational  Awareness,  here  A  (T)  to  TM  (T).  Here, 
again,  they  propose  an  exponential  relationship:  A  (T)  =  eK  +  ™(T)  /  ]  +  gK  +  tm(T)  ^  w^ere 
K  is  a  parameter  representing  the  product  of  many  value  added  factors  including  the 
completeness  of  the  COP.  Since  the  COP/  not-COP  condition  is  represented  in  the  Perry 
model  by  the  parameter  K,  we  have  chosen  to  set  K  =  .98  for  COP  and  K  =  .02  for  not- 
COP  to  attempt  to  achieve  a  large  spread  effect  for  Figure  3  below.  The  exact  value  for 
K,  interpreted  here  as  completeness  of  the  COP,  was  not  measured  beyond  the  ordinal 
scale  in  this  experiment,  although  it  is  entirely  feasible  to  take  more  precise 
measurements  of  completeness  of  the  COP  in  future  experiments. 


Figure  3.  Dynamic  Model  Relationship  between  Situational  Awareness,  A(T),  and 
Time  Spent  Collaborating  with  the  COP  (T). 


The  upper  curve  is  for  K  =.98  in  the  original  dynamic  model.  Setting  K  =  .02  shows  no 
apparent  impact  by  comparison,  and  this  curve  is  not  included  in  Figure  3.  The  lower 
curve  modifies  the  relationship  of  the  exponents  in  this  model  from  the  proposed 
summing  of  K  and  TM(T)  to  a  modified  version  involving  multiplication  of  the  two 
exponents,  which  effectively  lowers  the  overall  value  of  the  exponent.  As  the  reader  can 
see,  the  modified  version  of  the  dynamic  model  yields  a  somewhat  closer  fit  to  the 
experimental  data  points.  Never-the-less,  83  percent  of  the  12  experimental  data  points 
fall  below  60  percent  Situational  Awareness;  whereas  the  vast  majority  of  the  prediction 
points  from  either  of  the  model  versions  are  above  60  percent.  Thus  neither  version  of  the 
dynamic  model  fits  the  data  nearly  as  well  as  the  simple  linear  model  depicted  in  Figure 
1. 


Discussion 

Situational  Awareness  is  deemed  a  major  causal  factor  for  improved  combat 
effectiveness  for  the  warfighter  involved  in  Network  Centric  Operations.  We  have 
demonstrated  here  through  controlled  experimentation  that  use  of  a  relatively  complete 
COP  by  members  of  a  warfighting  team  causes  improved  Situational  Awareness  and  that 
this  Situational  Awareness  is  further  improved  in  proportion  to  the  amount  of  time  the 
warfighting  team  spends  in  collaboration  using  the  COP.  A  linear  “Tech  Team”  model  of 
Situational  Awareness  which  fits  these  data  has  been  advanced  here;  and  it  accounts  well 
for  the  findings  within  the  short  time  span  of  collaboration  examined.  It  seems  obvious 
that  there  exists  a  point  in  collaboration  time  where  further  time  spent  in  collaboration  by 
the  warfighting  team  would  have  diminishing  effects  on  improving  Situational 


Awareness.  Such  an  inflection  point  is  best  ascertained  by  further  controlled  empirical 
investigations. 


While  we  have  experimentally  demonstrated  a  couple  of  causes  of  Situational 
Awareness,  as  Perry  and  others  have  pointed  out,  there  undoubtedly  remain  several  other 
determinants  of  Situational  Awareness  (See  Perla,  2000).  The  Perry  value  added  model  is 
quite  comprehensive  and  includes  factors  on  the  technical  transmission  of  quality 
information  to  the  warfighting  team  as  well  as  additional  social/cognitive  factors  such  as 
different  “feature  vectors”  comprising  the  COP  and  the  varying  individual  interpretive 
capabilities  of  team  members. (  Perry  et  al,  2004).  Furthermore,  moving  from  individual 
Situational  Awareness  to  Shared  Situational  Awareness  requires  additional  modeling 
considerations  combined  with  controlled  empirical  investigation.  (See  Hiniker,  2000; 
Hiniker,  2002;  and  Perry  et  al,  2004). 

One  additional  factor  which  undoubtedly  impacts  Situational  Awareness  is  information 
overload.  As  the  number  of  warfighting  platforms,  red,  blue  and  neutral,  involved  in  a 
combat  scenario  increases  and  as  their  speed  and  complexity  of  movement  over  time 
increase,  the  capacity  of  any  warfigher  to  comprehend  the  situation  will  at  some  point  be 
exceeded,  and  his  Situational  Awareness  will  suffer  as  a  consequence.  We  are  developing 
new  information  technologies  for  the  warfighter,  such  as  Blue  Force  Tracker  and  the  User 
Defined  Operational  Picture.  All  these  technologies  will  bring  more  detailed  information 
to  the  warfighter.  Where  is  the  point  at  which  the  warfighter  becomes  overloaded?  And 
how  does  it  vary  across  warfighters  and  teams  of  warfighters  engaged  in  Network  Centric 
Operations? 
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